Integration of Audiovisually Compatible and Incompatible Consonants In Identification Experiments

Although studies of speech perception often focus on the interpretation of the acoustic speech waveform, in many situations the face of the talker can be seen by the listener and the perception of facial actions can greatly influence the interpretation of this signal. When the acoustic signal is unclear, observed facial actions can improve intelligibility [14]. On the other hand, even when the acoustic signal is highly intelligible, observing mismatched facial actions can reduce the effectiveness of acoustic speech cues. In the “McGurk Effect” [8, 7], a classic example of the latter influence, an artificial stimulus – constructed from the acoustic component of one consonant and the facial actions of another consonant that differs in place of articulation – may be perceived as a third consonant or as a consonant cluster.